The AI Infrastructure Alliance is developing a canonical stack for artificial intelligence and machine learning, bringing together a number of vendors, communities, and other organizations. As businesses seek to bring artificial intelligence and machine learning (AI/ML) into the mainstream, challenges emerge. Scaling from pilot projects to production can be difficult. Earlier this year, the issue got new attention and a group that seeks to address the problems got noticed. "Band of AI startups launch'rebel alliance' for interoperability" That headline in VentureBeat certainly caught my eye.

You might think that's something like Kubeflow but Kubeflow is more of a pipelining and orchestration system that's not really agnostic to the languages and frameworks that run on it.

We're already talking with several advanced data science engineering teams that are working on amazing open source projects that form the glue between different platforms, and we're looking to roll them under the Alliance." Of course, with so many moving parts to coordinate, fostering these emerging links hasn't been without challenges, and the AIIA is looking to learn from the missteps of similar precedents so that they can avoid making the same mistakes. "We've got to make sure that everyone sees the bigger picture and works together -- a rising tide lifts all boats," said Jeffries. "We don't want this to turn into a meaningless reference architecture. We don't want everyone in the Alliance pushing and pulling so hard that it warps the stack all out of proportion or collapses to individual interests. The trick here is to focus on mutual benefits -- every member of the Alliance must ask themselves how the Canonical Stack can help the Alliance as a whole. We also don't want governance by pure committee.

More than 20 AI startups have banded together to create the AI Infrastructure Alliance in order to build a software and hardware stack for machine learning and adopt common standards. The alliance brings together companies like Algorithmia; Determined AI, which works with deep learning; data monitoring startup WhyLabs; and Pachyderm, a data science company that raised $16 million last year in a round led by M12, formerly Microsoft Ventures. A spokesperson for the alliance said partner organizations have raised about $200 million in funding from investors. Dan Jeffries, chief tech evangelist at Pachyderm, will serve as director of the alliance. He said the group began to form from conversations that started over a year ago.